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ARTICLE 

Investigating University Educators' Design Thinking and 
the Implications for Design Support Tools 

Sue Bennett*, Shirley Agostinho* and Lori Lockyer f 


All university educators perform design work as they prepare and plan learning experiences for their 
students. How such design work is undertaken, conceptualised, and optimally supported is the focus of 
ongoing research for the authors. The purpose of this article is to present the results of a research study 
that sought to gain a richer understanding of university educators' design work; investigate how the idea 
of Learning Design could support design work; and examine how learning designs could be made available 
within a Learning Management System (LMS) as a design support tool. 

An overview of the outcomes from the entire research project is presented. The project's aims and 
outcomes and what was achieved are explained and potential future directions for this area of research 
are discussed. 
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Introduction 

The routine design work that all educators perform when 
preparing and planning learning experiences for students 
is an important part of their role. For university educators, 
designing effective learning experiences requires them 
to draw together their specialist domain expertise with 
appropriate teaching strategies, while integrating the 
range of digital technologies that are now commonplace 
in higher education. This represents a significant chal¬ 
lenge for even the most experienced university educators, 
one which has been supported by institutions and pro¬ 
fessional bodies, and through funding initiatives such as 
the UK's JISC (http://jisc.ac.uk/) and Australia’s Office for 
Learning and Teaching (http://www.olt.gov.au/). 

The field of Learning Design has developed a particular 
focus that is concerned with this routine design work done 
by educators to create learning experiences for their stu¬ 
dents. Learning Design refers to ways in which educators can 
document, model, implement, store, share, adapt and reuse 
pedagogical ideas. It has a particular focus on guidance (pro¬ 
viding tips and advice to educators), representation (docu¬ 
menting pedagogical ideas in consistent way) and sharing 
(enabling educators to build on the work of others by 
reusing and adapting pedagogical ideas). Learning Design 
has emerged as a particular branch of educational research 
and development that seeks to understand and support the 
design processes inherent in teaching. This body of work is 
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underpinned by the premise that supporting university edu¬ 
cators as they conceptualise, plan and prepare a unit 1 they 
are to teach in an upcoming academic session will result in 
more coherent and engaging learning experiences for stu¬ 
dents, with flow-on improvements in learning outcomes. 

This premise for design support has generated a mul¬ 
titude of different approaches, contributing to differ¬ 
ent aspects of Learning Design (see the Learning Design 
Conceptual Map, Figure 4 in Dalziel et al. 2013). Some 
have focused on developing tools to support design by 
helping educators think about and make decisions about 
their design ideas (e.g., The Learning Design Support 
Environment (LDSE) project explained in Laurillard et al. 
(2013); and see Conole (2013) for a review of Learning 
Design tools). Others have sought to support the shar¬ 
ing of ‘good’ design ideas through repositories or online 
networking tools (e.g., online community sharing (http:// 
cloudworks.open.ac.uk/), and collections of examples or 
cases (e.g., http://www.pedagogicalpatterns.org/, http:// 
www.learningdesigns.uow.edu.au/). These efforts have 
been complemented by investigations into different ways 
in which designs can been effectively represented, e.g. 
see Agostinho, Harper, Oliver, Wills, and Hedberg (2008), 
Agostinho (2009), Agostinho (2011), Agostinho, Bennett, 
Lockyer, Jones & Harper (2013); Conole (2013); and 
McAndrew and Goodyear (2013). Another line of work 
has sought to understand more about the fundamental 
design processes educators adopt in an effort to identify 
the context in which design occurs, the types of decisions 
that are part of the design process, and where support 
might be best located (e.g. Bennett et al., 2011; 2015). 

The purpose of this chapter is to present the results of 
a research study that contributed across several aspects of 
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learning design work - namely to understand more about 
the nature of design work as part of the teaching cycle 2 , to 
identify factors in the learning environment and explore 
how they influence design decisions, and to inform the 
design of tools and specifications to support guidance, 
representation and sharing. The project was conducted 
by an Australian research team with technical input from 
the Australian developers of a learning management sys¬ 
tem (Janison Solutions) and researchers from the Open 
University of the Netherlands. The aims of the project 
were to: 

1. Advance understanding of university educators' de¬ 
sign practices by interviewing university educators 
about how they undertake design (Investigation 1); 

2. Improve the methods used for selecting and 
representing high quality learning designs from 
real-life cases by reviewing an existing learning 
design repository and developing a refined learning 
design representation (Investigation 2); and 

3. Determine the feasibility of integrating learning 
design specifications (IMS-LD) into design support 
tools within the learning management system 
(Investigation 3). 

Below is an overview of the entire project that provides 
the background to our work, followed by an explanation 
of each of the three main investigations (aligned to the 
project aims as seen above) and a discussion of the key 
outcomes. A reflection on the outcomes is presented in 
terms of future directions for research and practice in this 
area. 

Background 

The idea for this study developed from research con¬ 
ducted by the authors between 2002 and 2005, and prior 
to that from their involvement in the Australian Univer¬ 
sity Teaching Committee (AUTC) project, ICTs and Their 
Role in Flexible Learning (2000-2002). The AUTC Learning 
Design project collected and described examples of teach¬ 
ing practice in higher education that made effective use 
of Information and Communication Technologies (ICTs). 
The outcome was an online repository of 32 exemplars 
derived from real-life cases. These cases are contextual¬ 
ised whilst there also is a smaller number of abstracted 
designs developed from 5 of the cases 3 . The intention was 
to provide these exemplars to educators as a stimulus 
for their own designs, so that they could tailor a learning 
design according to their particular circumstances and the 
demands of their contexts. 

One of the most significant contributions of the AUTC 
Learning Design project was the method developed to 
represent each exemplar, which combined graphical nota¬ 
tions of the sequence of tasks, resources and supports, 
with a textual description of the features of the design, 
the pedagogical reasoning underpinning it, the context in 
which it had been applied, and any evaluation/research 
outcomes (see Agostinho et al. (2008) for a detailed expla¬ 
nation of this learning design representation). The devel¬ 
opment of this approach to systematically documenting a 
learning design was not the first attempt at this goal and 


was one of a number of alternative approaches developed 
at the time (Agostinho, 2009). 

As a teaching and learning grant, the AUTC Learning 
Design project had a practical rather than research 
focus, and when the project was completed there were 
many questions about whether and how this ‘new’ 
method of representing learning designs would work. 
This became the focus for several subsequent research 
projects conducted by the authors, some of which are 
outlined below. 

For example, some further work undertaken by the 
authors was conducted as part of an interdisciplinary pro¬ 
ject, funded by the Smart Internet Cooperative Research 
Centre (2002-2004). The focus of this research and devel¬ 
opment work was to examine how metadata standards 
that were prominent at the time as a way of classifying 
learning objects (e.g., Learning Object Metadata (IEEE, 
2002)) could be used to help educators select appropri¬ 
ate learning designs and incorporate relevant learning 
objects. A further aim was to investigate the potential 
of documenting the resulting designs using technical 
standards that were new at the time, for example, the IMS 
Learning Design specification (Koper & Tattersall, 2005). 
The project was a collaboration between educational 
researchers (the authors) and researchers from informat¬ 
ics and engineering. Although the project outcomes were 
mainly focused on developing the metadata and techni¬ 
cal standards, (e.g. see Agostinho, Bennett, Lockyer, and 
Harper, 2004), this research work continued the authors’ 
interest in learning designs as a means of supporting edu¬ 
cators’ design processes. During the same period, work 
was undertaken on several smaller projects that sought to 
develop the learning design approach further by applying 
the method to representing designs relevant to the school 
sector, and working with school and university educators 
to test the application of learning designs in practice 
(e.g. see Bennett, Agostinho, and Lockyer, 2005). 

The culmination of this research work led to a concep¬ 
tualisation of learning design support tools that formed 
the basis for the research study discussed in this article. 
This conceptualisation is explained by the following sce¬ 
nario that illustrates how the research team envisaged the 
process by which an educator could use a learning design 
support tool to select a shared learning design, adapt it 
and implement it in their particular teaching context 
using any learning management system (LMS) and then 
possibly re-share the adapted learning design. 

1. Select a learning design: An educator reviews the 
learning designs contained in a repository and 
represented in a form that communicates the es¬ 
sential features of the design including a summary 
of the overall pedagogical approach. The learning 
design metadata (provided in the repository) assists 
the educator in identifying potentially appropri¬ 
ate designs to suit their context. For example, an 
educator might search for a particular pedagogical 
approach or learning outcome. 

2. Import the learning design into a LMS: After choosing 
a design from a repository, the educator imports 
the design into the learning management system. 
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This process is supported by technical specifications 
that provide standardised, machine-readable ways of 
describing learning designs. 

3. Customise the learning design: The educator uses 
their expert knowledge as a discipline specialist 
and teacher, drawing on knowledge of the context, 
customises the learning design in the LMS interface 
in the way they would normally work (rather 

than using a separate design tool). This process 
involves modifying the original design to introduce 
appropriate resources, providing specific details of 
tasks, and making decisions about how students will 
be supported to complete the tasks. Provision is also 
available for the addition of new tasks, resources 
and supports not present in the original design but 
deemed necessary by the educator. Over time, the 
educator develops the completed ‘unit of study’. 

This is a term that refers to a contextualised learning 
design, that is, one that includes information 
specific to the implementation context, such as 
specific content resources, dates for assessment 
submission, etc., as opposed to a more generic 
learning design that does not include context 
specific information. This distinguishes between 
a learning design that can be adapted (a generic 
learning design) and a learning design adapted 
for a particular context (a contextualised learning 
design or a ‘running learning design’, see glossary 
in Dalziel et al. 2013). 

4. Implement the unit of study. The unit of study is 
made available to students at the commencement 
of the academic session. After this point two types 
of teaching and learning activities occur. One 
type is the interactions between educator(s) and 
students on the site that are a normal part of the 
teaching and learning process. For example, the 
educator makes announcements, the educators(s) 
and students participate in online discussions, 
assignments are uploaded and feedback is 
provided. It is important to note here that the 
degree to which the actual interactions have been 
specified depends very much on the nature of the 
design. Some units may be highly specified prior 
to the commencement of the session, whereas 
others may adopt an open-ended design, one in 
which the actual activities are unspecified prior 
to implementation but instead are planned by 
the educator to unfold during implementation. 

This may be particularly so in project, problem, 
simulation, role-play or game-based designs. The 
second type of activity is the improvisation element 
of teaching, where it is desirable to make dynamic 
changes to adapt the design of the unit of study. 

For example, the educator might add major new 
resources, introduce a new learning support or 
change the nature of a task based on the responses 
of students. All of these possibilities highlight the 
potentially complex nature of learning design and 
demonstrate how design does not necessarily stop 
at the commencement of implementation (e.g. 

at the beginning of a teaching session), but can 


continue into the session such that by the end of 
session the design is different to that presented in 
the first week. Given that there can be limitations 
on what educators can change during a session as 
dictated by institutional policy, these changes may 
vary in significance from one institution or context 
to another. While significant redesign may be 
relatively rare, it should nevertheless be anticipated 
as a possibility in any learning design approach. 

5. Export a copy of the unit of study. At any stage of 
the above process the educator is able to export 
their unit of study to archive, to store for future 
use (e.g. to refine in a subsequent year), to share 
as a coherent whole with others, or to transfer the 
design into another LMS. 

6. Share a revised learning design: At the end of a 
teaching session, in addition to exporting the unit 
in its entirety, it is possible that an educator might 
want to share their revised design with others 
whereby specific content and detail is removed 
and additional pedagogical advice added. Technical 
tools would be available in the LMS to enable 

the educator to convert their unit of study into a 
‘sharable learning design’ format. 

This conceptualisation distinguishes a ‘learning design’ 
that is shared as something created with the intention 
to be customised. We propose that a shareable learning 
design should describe the pedagogical framework with 
resources, tasks and supports being abstracted somewhat 
from the original context so as to make it more readily 
adaptable and understandable both within and across 
disciplines. A ‘unit of study’ (a running learning design) 
is the product of the design process which is fully speci¬ 
fied as a particular teaching experience, at first ready for 
students and educators to interact with and then changed 
through that interaction in ways that may or may not 
alter the underlying design. Thus, the creation of a unit 
of study may result in a new design or design variant that 
could be shared with others, rather than sharing the more 
fully formed unit of study which may be more unwieldy 
to repurpose. 

The research study reported in this article explored 
how this process could be implemented in an LMS. 
The research study was comprised of three phases and 
each phase addressed particular aspects of the scenario 
explained above. The next section explains each of these 
three project phases. 

Overview of the project and outcomes 

The project was conducted as three inter-linked inves¬ 
tigations, the nature of which and their outcomes are 
described below. 

Investigation 1: Educator design thinking and practices 

This investigation was concerned with learning more 
about university educators’ routine design practices. The 
purpose of this investigation was to gain a better under¬ 
standing of the context into which learning design sup¬ 
port tools would be embedded. Specifically, we wanted to 
learn more about how university educators go about their 
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design work, what influences their decisions and what 
supports they draw on to identify aspects of current prac¬ 
tice into which tools might integrate and further develop. 

Institutional human research ethics approval was 
obtained and participants were recruited with an initial 
invitation distributed via electronic mailing lists through 
the following four Australian professional organisations 
with a higher education teaching and learning focus: 
Higher Education Research and Development Society of 
Australasia; the Australasian Society for Computers in 
Learning in Tertiary Education; the Open and Distance 
Learning Association of Australia; and the Australian 
Association for Research in Education. Those who 
responded were asked to provide some basic information 
about the discipline in which they taught, the number of 
years they had been a university educator, and their prior 
experience with online technologies. Based on this criteria 
a purposive sample of 30 participants across 16 Australian 
universities was selected. To simplify the question of 
discipline we used three broad discipline groups - Arts, 
Sciences and Professions. This approach was based on the 
work of Becher and Trawler (2001), and Shulman (2005) 
that identifies key differences between these discipline 
groupings. 

Participants were interviewed mainly by telephone, 
with a small number of local participants interviewed in 
person. The interview was conducted according to a semi- 
structured protocol that ensured coverage of key ques¬ 
tions, but also allowed for the conversation to flow and for 
unanticipated issues to arise and be discussed. The dura¬ 
tion of the interviews was between 60 and 90 minutes, 
all were audio-recorded and then transcribed for analy¬ 
sis. Participants were sent a copy of their transcript for 
verification. 

Coding of the transcripts was undertaken by all six mem¬ 
bers of the research team. Firstly a sub-set of interviews 
was read and annotated, each by one team member. The 
annotations were collated into codes, the set of codes was 
added to a framework developed from the research ques¬ 
tions. The framework was constructed in a table with mul¬ 
tiple columns that included the code name, a definition 
for each category and code, and example quotes. An addi¬ 
tional category called ‘emerging codes’ was established 
to capture any further relevant but unanticipated issues. 
With the coding framework developed, each transcript 
was allocated to two team members who coded it sequen¬ 
tially. This ensured that all coding was checked and where 
discrepancies arose could be resolved by changes to cod¬ 
ing or refinement of the coding framework. Coding was 
completed when all transcripts had been coded, checked 
and no further disagreement was detected. After coding 
was complete, the interview excerpts under each code 
were further analysed and interpreted thematically. 

The study found that our participants had a high degree 
of freedom when designing units. Where constraints 
existed they consisted of specific content or types of pro¬ 
fessional experience required by accrediting bodies, or 
requirements governing teaching practices, such as limits 
on the length, weighting or timing of assessment. There 
was a tendency for participants to work alone on the design 
of a unit they coordinated, while collaboration tended to 


occur more at the higher level of program planning. The 
nature of the context suggests there are opportunities for 
Australian university academics to innovate in their teach¬ 
ing without significant restrictions on their design deci¬ 
sions (see Bennett et al., 2011 for a detailed explanation). 

In terms of how university educators engage in the pro¬ 
cess of design, our participants began at different points 
depending on the context for their design. When design¬ 
ing a new unit, participants started from either an out¬ 
comes or content focus. When redesigning an existing 
unit, the starting point was based on what needed to be 
modified as the overall learning outcomes and content 
were already established. A second feature was that the 
process of design moves from broad to specific. The over¬ 
all framework of a unit in terms of the learning outcomes, 
content, and assessment, is usually designed first, followed 
then by designing/redesigning the more specific aspects 
of the unit such as weekly tutorial activities. A third fea¬ 
ture that emerged was that design is an iterative process 
that occurs before, during and after unit implementation. 
Participants explained how they engage in design before 
a unit is implemented in order to prepare the unit, but 
also how they think about design during unit implemen¬ 
tation in terms of designing specific resources or materi¬ 
als for the unit and reflecting on the progress of the unit, 
and how they continue to design after unit implementa¬ 
tion as part of reflecting how the unit can be modified 
for its next iteration. (These findings were discussed in a 
symposium - Goodyear et al., 2010). 

Four themes emerged from discussion about the fac¬ 
tors that influenced university educators' design prac¬ 
tices (see Bennett et al., 2015). A desire to meet learner 
needs was raised as an important consideration, with 
participants making key design decisions based on an 
understanding of the nature of the learner cohort. This 
included considering learners’ prior knowledge, their cul¬ 
tural backgrounds, their commitments beyond university 
or their need to be prepared for particular professions. 
Participants recognised that their designs were influenced 
by their past teaching experiences - particularly of suc¬ 
cessful or unsuccessful teaching strategies - but also 
ideas arising from formal study, professional learning and 
disciplinary practices. Experimentation and innovation 
were also drivers of design decisions, with accounts given 
of looking for and trying out new strategies found in the 
literature or shared by colleagues. All explained the need 
to work within the constraints of the university teaching 
context, which included university policies, but also the 
limitations of timetabling, workload, teaching spaces and 
resourcing. 

The study also sought to identify the support mecha¬ 
nisms academics use to develop their teaching practice. 
The participants in our study accessed a range of sources 
to generate and develop their ideas about teaching and the 
design of their units, including academic literature, work¬ 
shops, conferences, and informal discussions with col¬ 
leagues. Overwhelmingly, participants drew support from 
the ideas of ‘others’. This highlights the value academics 
place on the ideas of other educators, generally close col¬ 
leagues or people they see as similar to themselves. Our 
participants did not limit their interest exclusively to 
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those within their own discipline or in closely related dis¬ 
ciplines. However, they did place greater value on those 
actively engaged in teaching rather than those who may 
not have a direct teaching role. These findings support the 
concept of learning designs which provide contextualised 
ideas from credible others. Participants also commented 
on how central support units were more important early 
in their careers, and became less relevant as they gained 
experience. This suggests a preference for different types 
of supports depending on an educator’s career stage. 

Together, these findings provide insights into the con¬ 
text in which university educators do their design work 
and about their approaches, processes and influences 
on their decisions. The results reveal a complex process 
of balancing opportunity and constraint. The nature of 
their practice has much in common with the character¬ 
istics of design identified in the broader design stud¬ 
ies literature (e.g. Razzouk and Shute, 2012) although 
important differences exist. One key difference is that 
university educators are participants in the teaching and 
learning experiences they have designed in ways special¬ 
ist educational/instructional designers are usually not. 
While this difference is clear from personal and anecdotal 
accounts, it has received little attention from research¬ 
ers. Furthermore, designs are rather ephemeral in nature, 
they exist as particular instances experienced by students 
and educators, but they can be re-visited and revised 
when used again, either by the original designer or by 
another educator to whom to unit has been allocated. 
Thus in terms of providing design support tools, the 
main insight from this investigation was a design support 
tool that could not only assist an educator in designing 
a unit before implementing it, but can make provision 
for access throughout the implementation of a unit to 
refine, add, change, and after unit implementation as a 
reflection tool to document what could be changed for 
the subsequent unit iteration. 

Investigation 2: Review of learning design 
representation 

The second investigation in the research project 
involved revisiting the learning design descriptions of 
the 32 exemplars developed from the AUTC Learning 
Design project to examine whether the way in which 
they are described and documented could be considered 
‘effective’ based on more recent research about effective 
learning design representations. The original exemplars 
were presented in a four-part structure: 

• A summary of the exemplar's purpose and function, 
details of the design team and links to any publica¬ 
tions about implementation and evaluation. 

• A detailed description of the tasks, resources and 
supports in graphical and text form. 

• A description of the implementation context, includ¬ 
ing intended learning outcomes and assessment 
strategies. 

• Reflections about the rationale for the pedagogical 
approach, development and implementation history 
of the exemplar, details of any evaluation research, 
and perceived quality of the exemplar. 


Since completion of the project in 2002, international 
research in learning design had advanced understanding 
of how learning design representations could support the 
sharing and reuse of pedagogical ideas, particularly in 
higher education. Several newer learning design represen¬ 
tations had emerged (see Agostinho, 2009 for a summary). 
In addition, technical developments such as the IMS-LD 
specification and software applications compliant with it 
had advanced. Given these developments, it was timely 
to review the literature to determine the characteristics 
that constituted an ‘effective learning design representa¬ 
tion’ and use these characteristics as criteria to review the 
32 examples to compare them with the more contempo¬ 
rary benchmarks of effective descriptions. 

Analysis of the international research between 2002 
and 2009 led to the identification of characteristics of an 
‘effective’ learning design description. The literature drawn 
on included: Britain (2004); Conole, Littlejohn, Falconer 
and Jeffrey (2005); Falconer and Littlejohn (2006); 
Falconer, Beetham, Oliver, Lockyer, and Littlejohn (2007), 
Falconer and Littlejohn (2009), Littlejohn, Falconer, and 
McGill (2008) (see Agostinho et al., 2009 for full details). 
The following three fundamental characteristics that 
would support reuse were identified: 

• The pedagogy must be clear and explicitly described; 

• A quality rating of some form, such as evaluative 
findings, should be included; and 

• Explicit guidance or advice about how the learning 
design could be reused should be provided. 

These characteristics formed the basis of criteria for an 
instrument that was developed to review the 32 examples. 
The instrument consisted often elements, each of which 
was rated on a 5-point scale from very poor to very good 
and accompanied by a qualitative comment: The first six 
elements focused on providing clear and explicit pedagogy 
by summarising the learning design and explaining its 
pedagogy and implementation context as well as detailing 
the tasks, resources and supports used. The next three ele¬ 
ments focused on evaluating the learning design’s ‘qual¬ 
ity’ by reviewing the description about the rationale for 
ICT use, the explanation of any evaluation findings, and 
whether any reflections on the implementation has been 
provided by the designer(s). The final element focussed 
on whether the learning design description provided any 
advice and or guidance about reuse. 

All 32 exemplar were reviewed using this instrument 
and six were found to meet the criteria to be considered 
as effective learning design descriptions: 

1. ‘Environmental Decision Making’ - ‘role play’ focus 
in the discipline areas of environmental science 
(Brierley, Hillman, Devonshire, & Funnell, 2002) 

2. ‘Mekong e-Sim’ - ‘role play focus in the discipline 
area of social science (McLaughlan et al., 2002) 

3. ‘Predict-Observe-Explain’ - ‘procedure development’ 
focus in the discipline of science (Kearney, 2002) 

4. ‘Research Methods Online' - ‘problem based 
learning’ focus in the discipline area of education 
(Angus & Gray, 2002), 
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5. ‘Real life cases in multimedia’ - ‘project/case based’ 
focus in the discipline of education (Bennett, 2002), 

6. ‘Generic skills development’ - ‘collaborative’ focus in 
the discipline are of information technology (Luca, 
2002 ). 

As a result of this investigation refinements were made 
to the format for learning design representations and 
the six exemplars identified as containing the informa¬ 
tion required were adapted to the revised format. Refine¬ 
ments included adding some more specific information 
about how the learning design could be customised and 
highlighting key resources and supports of the learning 
design. Appendix A illustrates the revised format for 
describing a learning design; Sections 4-Checklist and 
Section 5-Design and implementation Tips are additional 
components. The revised descriptions of the six learn¬ 
ing designs were then used in Investigation 3 to explore 
the possibilities for integrating learning design support 
tools into a learning management system. The designs 
themselves have also been used as resources to stimulate 
design ideas and discussion in workshops (e.g. Bennett, 
Agostinho, & Lockyer, 2014). 

Investigation 3 

The purpose of Investigation 3 was to explore the feasibil¬ 
ity of integrating technical standards into a learning design 
support tool within a learning management system, and 
to develop a preliminary design for such a tool. This work 
was conducted in collaboration with our industry partner, 
Janison Solutions. The researchers worked with program¬ 
mers and educational designers at Janison Solutions to 
develop workflows and screen mock-ups which could be 
underpinned by the IMS-LD specification. Additional dis¬ 
cussions with research colleagues at the Open University 
of the Netherlands helped to clarify and refine the strate¬ 
gies tested. 

Investigation 3 explored the possibility of integrating 
the IMS-LD specification into design tools for teaching. 
The thinking at the time was that specifications would 
make learning materials ‘technically interoperable’ (that 
is they would be readily transferrable from one compli¬ 
ant system to another) thereby improving reusability. 
There was considerable interest in this idea at the time, 
although there was little adoption by developers of learn¬ 
ing management systems. 

Rather than fully develop a software solution, the goal 
of the project was to explore the potential for integrating 
the IMS-LD specification and devise a strategy for how it 
might be achieved. In essence, the goal was ‘proof of con¬ 
cept’ rather than a developed product. The research team 
worked with the industry partner's technical and design 
staff to test a series of possible scenarios that would suit 
the overall philosophy of the project which was to allow 
for a high degree of customisation of the learning design 
by an educator. 

The team ultimately devised the following general pro¬ 
cess that would reflect the conceptualisation developed at 
the commencement of the project: 


1. The educator selects a learning design from those 
available in the repository and this appears as a 
partially completed unit of study in the learning 
management system. The technical specifica¬ 
tion allows the design to be ‘read into’ the system 
with the relevant system components appearing 
according to the design. The scaffold includes both 
pre-determined characteristics and customisable 
features. For example, a problem-based design may 
include a discussion activity as a key learning task 
(a pre-determined characteristic), but the educa¬ 
tor would be given the option of offering that 
discussion in face-to-face, synchronously online or 
asynchronously online modes. Thus, the starting 
point for building specifications comes from what is 
already known about the design chosen. 

2. The educator makes changes to the design within 
the learning management system to customise it to 
suit his/her own context. The flexible design tool 
allows an educator to start working at any point, 

at either a macro or micro level, to begin building 
a design based on the established over-arching 
framework. (This reflects what our interviews had 
revealed about the processes educators used already 
as part of their design practices.) The educator 
makes choices to specify the resources, tasks 
and supports to be used, and adds and removes 
elements as appropriate. When adding or removing 
new task, resources or supports that change the 
original design, the educator is prompted to add 
notes explaining the pedagogical rationale. Behind 
the scenes, the software builds the IMS-LD meta¬ 
data for the ‘unit of study’ that is being created, 
out of sight of the educator who interacts with 
the learning management system interface that 
is already familiar. Metadata relevant to IMS-LD 
is recorded, while additional metadata beyond 
the scope of the specification but useful to future 
sharing of the design is captured as separate notes. 
This approach solves key problems by constructing 
the metadata record without an educator 
interacting directly with the technical specification 
and removes to need for a separate design interface 
because the educator is working within the LMS. 
This approach also facilitates updates to the ‘unit of 
study’ dynamically during the teaching session such 
that the metadata is also updated and changes to 
the design are captured. 

3. At any point in the process the educator is able to 
‘export’ their design from the learning management 
to back-up elsewhere or to share. IMS-LD metadata 
is exported and thus available for import into 

a compliant system, with options provided for 
exporting the ‘unit of study’ (running learning 
design) or the learning design. The former contains 
the specific detail of the unit, with options to 
include, for example, the content of discussion 
forums and student assignment submissions where 
desired. The latter includes only the structural 
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features of the design. Upon export, the system 
prompts the educator to add to the metadata, for 
example, with reflections on the implementation 
or results from evaluations that provide evidence of 
the efficacy of the design. 

Full development of a learning design support tool envis¬ 
aged as an outcome of this investigation was beyond the 
scope of this project and would require significant fur¬ 
ther funding. A key insight from this investigation was 
the need to include opportunities to collect additional 
data beyond that included in the IMS-LD specification to 
achieve the design supports and the outputs characteris¬ 
tic of the effective learning design representations identi¬ 
fied in Investigation 2. 

Discussion 

The findings from this study advance our understand¬ 
ing of university educators’ design thinking by providing 
insights into existing practice that are anecdotally famil¬ 
iar, but have garnered little empirical evidence to date. 
The accounts from our participants from Investigation 1 
suggest that there is an existing design practice that is 
part of routine teaching but this is under-developed 
in comparison to other types of design activity such as 
architecture and engineering. Perhaps because educa¬ 
tor’s design work is integrated into their broader teaching 
practice it has been paid much less attention than more 
easily delineated activities such as lecturing and assessing. 
We currently lack the vocabulary to discuss it clearly and 
have only recently begun the conceptual work which is 
needed to underpin the further of learning design sup¬ 
port tools. 

There is much more to learn, also, about effective rep¬ 
resentations. There is a theoretical and empirical basis 
for providing educators with solutions to past problems 
that are abstracted sufficiently from the original context 
to promote customisation (Kolodner, Owensby, & Guzdial, 
2004). Our own investigations have used a representa¬ 
tion that combines graphical and textual information, 
and chooses brevity over detail. But there is still little con¬ 
sensus in the learning design literature over the critical 
characteristics of effective representations. Our findings 
from Investigation 2 identified the inclusion of reflective 
and evaluative information, together with guidance and 
advice within learning design representations, as a key 
characteristic that would support reuse. This finding from 
our review of the literature needs further empirical inves¬ 
tigation. Findings from Investigations 1 and 2 suggest 
that developing one’s own teaching practice by adapting 
ideas from ‘respected others’ is a strategy already used by 
educators looking to expand their repertoire. This is high¬ 
lighted from interview data in Investigation 1 and the 
literature reviewed as part of Investigation 2, and lead to 
the inclusion of a ‘quality’ rating from evaluative findings 
in the revised learning design description. But how this 
can be fostered, particularly within institutions by central 
units, needs careful thinking to cater for different disci¬ 
pline backgrounds and different career stages. 


What is clear from Investigation 1 is that tools to sup¬ 
port design decision-making must be flexible. Flexibility 
is needed because design is not only about planning 
and preparation prior to an academic session, but occurs 
throughout sessions, dynamically in response to learners’ 
emerging needs and extends beyond, as part of educators’ 
reflections on their experiences. Flexibility is also needed 
because design is iterative with various starting points, 
depending on the nature of the changes to be made, and 
with attention shifting between macro to micro features. 
Lock-step tools that restrict the order in which a design 
is specified are unlikely to be successful, but at the same 
time tools must help an educator navigate through a 
developing design. Learning design support tools that 
include these features will act as a coach, with the goal of 
empowering educators in the design process with good 
tools rather than correcting poor design or corralling the 
design process to narrowly. 

It is important to contextualise these interpretations and 
speculations within the limitations of this study. Firstly, the 
project was only concerned with investigating and refin¬ 
ing the learning design representation originating from 
the AUTC Learning Design project. This is only one of a 
number of representations attempting to address the same 
challenge. Investigation 1 was conducted in the Australian 
context and attracted volunteers with an interest in teach¬ 
ing and learning. Members of this self-selecting group are 
not necessarily representative of all university educators 
and so their experiences must be interpreted with this in 
mind. The study also took place at a particular point in 
time, and it must be recognised that ongoing changes to 
higher education bring new challenges. Given the increas¬ 
ing importance of online pedagogies, however, this is only 
likely to heighten the pressure on educators to design well. 

Conclusion 

High quality design is critical to effective learning experi¬ 
ences and outcomes. The context in which university edu¬ 
cators work is increasingly challenging with a more diverse 
student body than ever before and new technologies 
becoming integral to higher education. Effective design 
supports are needed and this is the challenge the field of 
Learning Design engages with. The findings of this project 
advance our thinking about the need for learning design 
support tools to include flexibility that enables an educator 
to import, adapt, revise, refine and reflect on their design 
before, during and after implementation. At the same time, 
our findings expose the magnitude of the challenge. The 
core concepts of Learning Design - guidance, representa¬ 
tion and sharing - offer rich opportunities for further theo¬ 
retical, empirical and practical work and already encompass 
a wide range of approaches and initiatives that can be built 
upon as this emerging field develops. As part of this devel¬ 
opment, the field must engage more with educators to test 
and refine their ideas and convince university administra¬ 
tors and policy-makers of the significance of this approach. 
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Appendix A 

Investigation 2 - Revised format for learning design representations. 


LD title 


1. Overview 

a. Brief description 


b. Graphical representation 


2. General Information 

a. Learning Objectives 


b. Pedagogical Rationale 


c. Evidence of Quality 


3. Textual description of design sequence 

a. Resources/Tasks/Supports 


b. Suggestions for assessment 


4. Checklist 

a. Resources 

□ Critical resources listed here 

b. Supports 

□ Critical supports listed here 

5. Design and Implementation Tips 

a. Customising design 


b. Set-up required before implementation 


c. Implementation ideas 


6. Acknowledgements 



7. References 
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Notes 

1 The term ‘unit’ is used generically throughout to refer 
to a module, session or learning activity that an educa¬ 
tor designs for students to engage with. 

2 Italics refer to specific components of the Learning 
Design Conceptual Map, Figure 4 in Dalziel et al. 2013. 

3 www.learningdesigns.uow.edu.au 
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